Measuring Voter Registration and Turnout in Surveys Do Official Government Records Yield More Accurate Assessments?

In many post-election surveys, the proportion of respondents who claim to have voted is greater than government-reported turnout rates. These differences have often been attributed to respondent lying (e.g., Burden 2000). In a search for greater accuracy, scholars have replaced respondent self-reports of turnout with government records of their turnout (a.k.a. turnout validation). Some scholars have interpreted “validated” turnout estimates as more accurate than respondent selfreports because “validated” rates tend to be lower than aggregate selfreported rates and tend to be closer to government-reported rates. We explore the viability of turnout validation efforts. We find that several Matthew K. Berent is the president of Matt Berent Consulting, Sharon, PA, USA, and consultant to the Institute for Social Research at the University of Michigan, Ann Arbor, MI, USA, and to the Institute for Research in the Social Sciences at Stanford University, Stanford, CA, USA. Jon A. Krosnick is professor of communication, political science, and psychology at Stanford University, Stanford, CA, USA, and University Fellow at Resources for the Future, Washington, DC, USA. Arthur Lupia is a professor of political science, and research professor, Institute for Social Research at the University of Michigan, Ann Arbor, MI, USA. The authors thank Logan Casey, Donald Green, Michael Hamner, Kristyn Karl, Spencer Piston, Michael Traugott, and Nicholas Valentino for helpful suggestions on earlier versions of this manuscript. The authors also thank Michael McDonald for his help in acquiring and understanding government records, Matthew DeBell for his technical advice, Eitan Hersh for information about deadwood in government records, Jeffrey Lewis for directing them to the Centers for Disease Control list-matching software, and Darrell Donakowski for his advice on where to find relevant data. Finally, the authors also thank Bob Blaemire, Brian Stults, and Andrew Tavani for information about list matching. This work was supported by grants from the National Science Foundation [SES0535332 to J.A.K., SES-0535334 to A.L.]. This paper draws from data and procedures described in an American National Election Studies technical report entitled “The Quality of Government Records and Over-Estimation of Registration and Turnout in Surveys: Lessons from the 2008 ANES Panel Study’s Registration and Turnout Validation Exercises” (http://www.electionstudies.org/resources/papers/nes012554.pdf). *Address correspondence to Jon A. Krosnick, 432 McClatchy Hall, Stanford University, Stanford, CA 94305, USA; e-mail: [email protected]. Public Opinion Quarterly doi:doi:10.1093/poq/nfw021 Public Opinion Quarterly Advance Access published August 2, 2016 at Stnford U niersity on A uust 2, 2016 http://poqrdjournals.org/ D ow nladed from apparently viable methods of matching survey respondents to government records severely underestimate the proportion of Americans who were registered to vote. Matching errors that severely underestimate registration rates also drive down “validated” turnout estimates. As a result, when “validated” turnout estimates appear to be more accurate than self-reports because they produce lower turnout estimates, the apparent accuracy is likely an illusion. Also, among respondents whose selfreports can be validated against government records, the accuracy of self-reports is extremely high. This would not occur if lying was the primary explanation for differences between reported and official turnout rates. These findings challenge the notion that the practice of “turnout validation” offers a means of measuring turnout that is more accurate than survey respondents’ self-reports.